DeltaTree: A Practical Locality-aware Concurrent Search Tree

As other fundamental programming abstractions in energy-efficient computing, search trees are expected to support both high parallelism and data locality. However, existing highly-concurrent search trees such as red-black trees and AVL trees do not consider data locality while existing locality-aware search trees such as those based on the van Emde Boas layout (vEB-based trees), poorly support concurrent (update) operations. This paper presents DeltaTree, a practical locality-aware concurrent search tree that combines both locality-optimisation techniques from vEB-based trees and concurrency-optimisation techniques from non-blocking highly-concurrent search trees. DeltaTree is a $k$-ary leaf-oriented tree of DeltaNodes in which each DeltaNode is a size-fixed tree-container with the van Emde Boas layout. The expected memory transfer costs of DeltaTree's Search, Insert, and Delete operations are $O(\log_B N)$, where $N, B$ are the tree size and the unknown memory block size in the ideal cache model, respectively. DeltaTree's Search operation is wait-free, providing prioritised lanes for Search operations, the dominant operation in search trees. Its Insert and {\em Delete} operations are non-blocking to other Search, Insert, and Delete operations, but they may be occasionally blocked by maintenance operations that are sometimes triggered to keep DeltaTree in good shape. Our experimental evaluation using the latest implementation of AVL, red-black, and speculation friendly trees from the Synchrobench benchmark has shown that DeltaTree is up to 5 times faster than all of the three concurrent search trees for searching operations and up to 1.6 times faster for update operations when the update contention is not too high

Scalability of Distributed Version Control Systems

Source at https://ojs.bibsys.no/index.php/NIK/article/view/434.Distributed version control systems are popular for storing source code, but they are notoriously ill suited for storing large binary files. We report on the results from a set of experiments designed to characterize the behavior of some widely used distributed version control systems with respect to scaling. The experiments measured commit times and repository sizes when storing single files of increasing size, and when storing increasing numbers of single-kilobyte files. The goal is to build a distributed storage system with characteristics similar to version control but for much larger data sets. An early prototype of such a system, Distributed Media Versioning (DMV), is briefly described and compared with Git, Mercurial, and the Git-based backup tool Bup. We find that processing large files without splitting them into smaller parts will limit maximum file size to what can fit in RAM. Storing millions of small files will result in inefficient use of disk space. And storing files with hash-based file and directory names will result in high-latency write operations, due to having to switch between directories rather than performing a sequential write. The next-phase strategy for DMV will be to break files into chunks by content for de-duplication, then re-aggregating the chunks into append-only log files for low-latency write operations and efficient use of disk space

Uncertainty Estimation and Visualization of Wind in Weather Forecasts

The Collaborative Symbiotic Weather Forecasting system, CSWF, let individual users do on-demand small region, short-term, and very high-resolution forecasts. When the regions have some overlap, a symbiotic forecast can be produced based on the individual forecasts from each region. Small differences in where the center of the region is located when there is complex terrain in the region, leads to significant differences in the forecasted values of wind speed and direction. These differences reflect the uncertainty of the numerical model. This paper describes two different ways of presenting these differences using a traditional map based approach on a laptop and a display wall, and an augmented reality approach on a tablet. The approaches have their distinct advantages and disadvantages depending on the actual use and requirements of the user

Scalability of Distributed Version Control Systems

Distributed version control systems are popular for storing source code, but they are notoriously ill suited for storing large binary files. We report on the results from a set of experiments designed to characterize the behavior of some widely used distributed version control systems with respect to scaling. The experiments measured commit times and repository sizes when storing single files of increasing size, and when storing increasing numbers of single-kilobyte files. The goal is to build a distributed storage system with characteristics similar to version control but for much larger data sets. An early prototype of such a system, Distributed Media Versioning (DMV), is briefly described and compared with Git, Mercurial, and the Git-based backup tool Bup. We find that processing large files without splitting them into smaller parts will limit maximum file size to what can fit in RAM. Storing millions of small files will result in inefficient use of disk space. And storing files with hash-based file and directory names will result in high-latency write operations, due to having to switch between directories rather than performing a sequential write. The next-phase strategy for DMV will be to break files into chunks by content for de-duplication, then re-aggregating the chunks into append-only log files for low-latency write operations and efficient use of disk space

Flexible Devices for Arctic Ecosystems Observations

Devices for observing the environment range from basic sensor systems, like step-counters, through wild-life cameras, with limited processing capabilities, to more capable devices with significant processing, memory and storage resources. Individual usage domains can benefit from a range of functionalities in these devices including flexibility in prototyping, on- device analytics, network roaming, reporting of data, and keeping the devices and services available in spite of failures and disconnections. The problem is that either the devices are too resource limited to support the range of functionalities, or they use too much energy. An important usage domain is COAT – Climate-Ecological Obser- vatory for Arctic Tundra. Presently, best practice includes deploying wild-life cameras in the Arctic Tundra, and visiting them to manually collect the recorded observations. This is a problem because such devices can only be rarely visited, and manual approaches to fetching data and storing it do not scale with regards to number of cameras, handling of human mistakes, and with freshness of observations. We present a prototype for observing the environment composed of a general purpose computer, a Raspberry PI, in combination with an ARM-based microcontroller. The combination enables us to create a more energy efficient prototype while supporting the needed functionality. The prototype improves on currently applied methods of observing the Arctic tundra. The prototype automatically observes the arctic tundra through camera, humidity and temperature sensors. It monitors itself for failures. The data is stored locally on the prototype until it can be automatically reports to a backend service over a wireless network. We have conducted experiments that show that task scheduling can reduce power consumption, and we identify some additional points that need to be addressed before we can run the device for long periods on battery power

Embarrassingly Distributed Computing for Symbiotic Weather Forecasts

This article is part of Bård Fjukstad's doctoral thesis, which is available at http://hdl.handle.net/10037/6961</a

DeltaTree: A Practical Locality-aware Concurrent Search Tree

As other fundamental programming abstractions in energy-e cient computing, search trees are expected to support both high parallelism and data locality. However, existing highly-concurrent search trees such as red-black trees and AVL trees do not consider data locality while existing locality-aware search trees such as those based on the van Emde Boas layout (vEB-based trees), poorly support concurrent (update) operations. This paper presents DeltaTree, a practical locality-aware concurrent search tree that combines both locality-optimisation techniques from vEB-based trees and concurrency-optimisation techniques from non-blocking highly-concurrent search trees. DeltaTree is a k-ary leaf-oriented tree of DeltaNodes in which each DeltaNode is a size- xed tree-container with the van Emde Boas layout. The expected memory transfer costs of DeltaTree's Search, Insert and Delete operations are O(logBN), where N;B are the tree size and the unknown memory block size in the ideal cache model, respectively. DeltaTree's Search operation is wait-free, providing prioritised lanes for Search operations, the dominant operation in search trees. Its Insert and Delete operations are non-blocking to other Search, Insert and Delete operations, but they may be occasionally blocked by maintenance operations that are sometimes triggered to keep DeltaTree in good shape. Our experimental evaluation using the latest implementation of AVL, red-black, and speculation friendly trees from the Synchrobench benchmark has shown that DeltaTree is up to 5 times faster than all of the three concurrent search trees for searching operations and up to 1.6 times faster for update operations when the update contention is not too high

Quantifying the variability of power and energy consumption for IoT edge nodes

For IoT and edge systems, measuring, predicting and optimizing energy consumption is an open field. It is important to accurately and precisely characterize power and energy consumption of edge nodes, as energy can be a scarce and key resource. However, there are no fine-grain studies that aim at understanding the potential variability of power and energy consumption of edge nodes. Existing research works give minor or no significance to this potential variability. This paper addresses this problem by quantifying the variability of power and energy consumption on a single edge node, and among multiple homogeneous edge nodes, for three scenarios: Idle, CPU intensive “matrix product” and RAM intensive “flip”. These scenarios are found in edge applications. Identical controlled experiments are repeated thoroughly, for each scenario. Results show that power and energy variability exist for all studied scenarios. On a single node, power and energy variability measurements are relatively low. On multiple homogeneous nodes, the variability can be significant. For example, for CPU intensive “matrix product”, the variability in energy is equivalent to an idle up-time, in a month, of 7 hours and 52 hours, when considering single and multiple homogeneous nodes, respectively

Flexible Devices for Arctic Ecosystems Observations

Source at http://ojs.bibsys.no/index.php/NIK/article/view/435.Devices for observing the environment range from basic sensor systems, like step-counters, through wild-life cameras, with limited processing capabilities, to more capable devices with significant processing, memory and storage resources. Individual usage domains can benefit from a range of functionalities in these devices including flexibility in prototyping, on- device analytics, network roaming, reporting of data, and keeping the devices and services available in spite of failures and disconnections. The problem is that either the devices are too resource limited to support the range of functionalities, or they use too much energy. An important usage domain is COAT – Climate-Ecological Obser- vatory for Arctic Tundra. Presently, best practice includes deploying wild-life cameras in the Arctic Tundra, and visiting them to manually collect the recorded observations. This is a problem because such devices can only be rarely visited, and manual approaches to fetching data and storing it do not scale with regards to number of cameras, handling of human mistakes, and with freshness of observations. We present a prototype for observing the environment composed of a general purpose computer, a Raspberry PI, in combination with an ARM-based microcontroller. The combination enables us to create a more energy efficient prototype while supporting the needed functionality. The prototype improves on currently applied methods of observing the Arctic tundra. The prototype automatically observes the arctic tundra through camera, humidity and temperature sensors. It monitors itself for failures. The data is stored locally on the prototype until it can be automatically reports to a backend service over a wireless network. We have conducted experiments that show that task scheduling can reduce power consumption, and we identify some additional points that need to be addressed before we can run the device for long periods on battery power.</p